Importing the required libraries¶
In [1]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import os
import math
import plotly.graph_objects as pl
Loading the dataset¶
In [2]:
#Getting the current working directory
os.getcwd()
Out[2]:
In [3]:
#Changing the working directory location to load the dataset.
os.chdir('/Users/aishwaryamaddimsetty/Downloads')
In [4]:
#Importing the dataset from my local computer and assigning it to a variable:
data = pd.read_csv('DSA_DataSet.csv')
Studying the dataset - basic summary statistics, understanding what the dataset comprises etc.¶
In [5]:
#Printing the first 5 rows of the dataset - just a look at the data
data.head(5)
Out[5]:
In [6]:
#Printing the number of rows and columns of the dataset:
data.shape
Out[6]:
In [7]:
#Printing the details about the columns of the dataset:
data.info()
This shows the data has no missing values so no handling required for missing data in data cleaning part¶
In [8]:
#Printing the datatypes of the columns:
data.dtypes
Out[8]:
In [9]:
#Printing the basic summary statistics of the dataset:
data.describe()
Out[9]:
In [10]:
#Printing the number of nulls in the dataset (though we know from above step(data.info) we have all non null columns):
data.isnull().sum()
Out[10]:
In [11]:
#Small code to get counts of each category present in the data's catogorical columns:
category_column = [i for i in data.columns if data[i].dtypes == 'object']
for column in category_column:
print(column, '\n\n')
print(data[column].value_counts())
print("---" *20)
In [12]:
#Used the term 'target' as the variable, since that is our result
target_count = data['y'].value_counts()
target_count
Out[12]:
Can infer from above analysis that the data is imbalanced, given that "Nos" are 88.73% of the dataset compared to the "Yes's".¶
Exploratory Data Analysis (EDA)- Data Visualization¶
In [13]:
#Analysing the column ('y') visually - Yess and Nos
colors = ['Red', 'Green']
trace = pl.Pie(labels =target_count.index, values = target_count.values, pull= [0.05], marker=dict(colors=colors))
layout = pl.Layout(title = "Subscribed to the Term Deposit", height = 200, legend= dict(x=1.1, y=1.3))
fig = pl.Figure(data=[trace], layout = layout)
fig.update_layout(height=500, width=600)
fig.show()
Visualization of the data based on data type of the columns¶
(Categorical or Numerical).¶
Bar charts to understand the distribution of values within the column¶
Countplots to understand the distribution of values with in the column¶
and corresponding to target column ( Subscribed or Not)¶
This univariate analysis is to understand, the column's distribution and insights on if any sub category within the column has a significance to subscribing or not.¶
In [14]:
# JOB
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['job'])
Out[14]:
From above graph, we can say, in the data, clients with admin, blue_collar and technician ctegory jobs are high compared to other job categories.¶
In [15]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='job', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right", size=15)
plt.tight_layout()
plt.show()
From above graph, we can infer that highest number of subscriptions are clients with admin job, next highest is technicians. But point here to be noted is that we also have a very high number of clients in that job category who did not subscribe. This is because simply there are more clients in our dataset who fall in that category of job.¶
In [16]:
# MARITAL
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['marital'])
Out[16]:
Simple insight is that the we have majority of this column's distribution in married category.¶
In [17]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='marital', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
In [18]:
#DEFAULT
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['default'])
Out[18]:
From the above graph, it looks like we barely have clients who have credit, major of them do not have credit.¶
In [19]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='default', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
In [20]:
#EDUCATION
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['education'])
Out[20]:
In [21]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='education', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
From above graphs on education, we can infer that, clients with university degree are a majority, and also highest number of subscribers are also clients with university degree.¶
In [22]:
# HOUSING
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['housing'])
Out[22]:
In [23]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='housing', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
We can see that, majority of the clients have a housing loan. One direction we can go for further analysis is that we dig deeper in clients with any kind of loans and if they have subscribed or not.¶
In [24]:
#LOAN
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['loan'])
Out[24]:
In [25]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='loan', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
We observe that there are more clients without a personal loan, who have subscribed. But we see that a huge number of clients with no personal loan also , din't subscribe to the deposit. So we can't really say that, peopel with and without loans hold any significance in subscribing or not.¶
In [26]:
#CONTACT
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['contact'])
Out[26]:
In [27]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='contact', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
In [28]:
#MONTH
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['month'])
Out[28]:
We can infer from plot above that, maximum number of clients were contacted in the month of May. ( Range can be set as May to August).¶
In [29]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='month', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
First observation for me is that , there has been very little activity in the month of december.(understandable that it's holiday month). We can say that, there are very few clients who have subscribed in comparision to number of people contacted in summer months. And, rough analysis can be said that in march, april, september and october are a little more succesive months in comparision to other months.¶
In [30]:
#DAY OF THE WEEK
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['day_of_week'])
Out[30]:
In [31]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='day_of_week', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
By first look of the plot, can infer that averagly number of clients who have subscribed are roughly same through out the week. irrespective of the day of contact, roughly 1000 clients have subscribed. So one underatdning is that, this column wont hold any significance in predicting, due to it's lack of any significance between sub categories.¶
In [32]:
#POUTCOME
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['poutcome'])
Out[32]:
In [33]:
plt.figure(figsize = (10,5))
ax = sns.countplot(x='poutcome', hue= 'y', data = data, palette = 'Set1')
ax.set_xticklabels(ax.get_xticklabels(), rotation = 40, ha = "right")
plt.tight_layout()
plt.show()
Analysis for this column is that, number of people who have previously subscribed have subscribed for this new term aswell. So it has some significance in predicting. One other insight is that majority of the customers are new customers and this is derived from the bar that shows, there is a huge nummber of clients for whom , previous outcome is unknown. hence ...¶
In [34]:
sns.boxplot(data = data, x = 'y', y = 'age', hue = "y")
plt.tight_layout()
In [35]:
plt.figure(figsize=(10,8))
sns.distplot(data["age"])
Out[35]:
As we can see in the above distribution also, that most of the customers are in the age range of 30-50. Histogram helps us understand the column distrubtion¶
In [36]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="duration")
plt.show()
From the above plot it is clear that, the duration (last contact duration) of a customer can be useful for predicting the target variable. It is expected because it is already mentioned in the data overview that this field highely affects the target variable and should only be used for benchmark purposes.¶
In [37]:
plt.figure(figsize=(10,8))
sns.distplot(data["duration"])
plt.show()
This seems like a powerlaw distribution where most the values are very low and very few have high values.¶
In [38]:
plt.figure(figsize=(12,10))
plt.subplot(2,1,1)
sns.countplot(x='campaign', hue = 'y', data = data, palette ='Set1')
plt.xlim(right = 10)
plt.xlabel('')
plt.subplot(2,1,2)
sns.countplot(x='campaign', hue = 'y', data = data, palette = 'Set1')
plt.xlim(left = 11)
plt.ylim(top = 30)
plt.xlabel('No of Campaigns', fontsize = 14)
plt.show()
This attribute tells us the number of time , teh client was contacted in this campaign. Can be observed that, more number of times the contact , less subscriptions.¶
In [39]:
sns.boxplot(data=data, x="y", y="campaign")
plt.show()
In [40]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["campaign"])
plt.show()
In [41]:
data['pdays'].unique()
Out[41]:
In [42]:
data['pdays'].value_counts()
Out[42]:
Most of the values are 999, which means that the most of the customers have never been contacted before¶
In [43]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="pdays")
plt.show()
In [44]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data[data["y"]=="yes"]["pdays"])
sns.distplot(data[data["y"]=="no"]["pdays"])
plt.show()
In [45]:
data["previous"].unique()
Out[45]:
In [46]:
data["previous"].value_counts()
Out[46]:
In [47]:
data[data["y"]=="yes"]["previous"].value_counts()
Out[47]:
In [48]:
data[data["y"]=="no"]["previous"].value_counts()
Out[48]:
In [49]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="previous")
plt.show()
In [50]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["previous"])
plt.show()
In [51]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data[data["y"]=="yes"]["previous"])
sns.distplot(data[data["y"]=="no"]["previous"])
plt.show()
The previous feature is very similarly distributed for both the classes in the target variable. From basic EDA it is not sure how much value this individual feature have on the target variable.¶
In [52]:
# Marital
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(data['previous'])
Out[52]:
In [53]:
# Easy count plots:
sns.countplot(x='previous', hue= 'y', data = data, palette= 'Set1')
plt.show()
In [54]:
data["emp.var.rate"].value_counts()
Out[54]:
In [55]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="emp.var.rate")
plt.show()
In [56]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["emp.var.rate"])
plt.show()
In [57]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="cons.price.idx")
plt.show()
In [58]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["cons.price.idx"])
plt.show()
In [59]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="cons.conf.idx")
plt.show()
In [60]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["cons.conf.idx"])
plt.show()
In [61]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="euribor3m")
plt.show()
In [62]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["euribor3m"])
plt.show()
In [63]:
%matplotlib inline
sns.boxplot(data=data, x="y", y="nr.employed")
plt.show()
In [64]:
%matplotlib inline
plt.figure(figsize=(10,8))
sns.distplot(data["nr.employed"])
plt.show()
Understanding the features well enough to do manual encoding ( changing few useful categorical columns to numeric)¶
Data Preprocessing for further modeling.¶
In [65]:
# Cleaning the data and making it ready for modeling.
In [66]:
# Importing the libraries required for machine learning and preprocessing the data:
import pandas as pd
import pickle
from sklearn import preprocessing
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.metrics import roc_auc_score
First step in data cleaning is handling the missing values, but seeing the steps above(data.info()), there are no nulls in this dataset.¶
In [67]:
# Dropping the model prediction column from the dataset
data = data.drop(columns = ['ModelPrediction'])
In [68]:
# Handling the duplicate data.
data_dups = data[data.duplicated(keep = "last")]
data_dups
Out[68]:
In [69]:
#Getting the count of duplicated values
data_dups.shape
Out[69]:
In [70]:
#### Looks like we have 12 duplicated rows. SO dropping the duplicate rows is adviced.
data = data.drop_duplicates()
In [71]:
data.shape
Out[71]:
Get the target (class variable) and independent features¶
In [72]:
data_x = data.iloc[:, :-1]
print("Shape of X: ", data_x.shape)
In [73]:
data_y = data['y']
print('Shape of Y: ', data_y.shape)
Splitting the data into train and test sets¶
In [74]:
X_rest, X_test, y_rest, y_test = train_test_split(data_x, data_y, test_size = 0.2)
X_train, X_cv, y_train, y_cv = train_test_split(X_rest, y_rest, test_size =0.2)
In [75]:
print("X Train:", X_train.shape)
print("X CV:", X_cv.shape)
print("X Test:", X_test.shape)
print("Y Train:", y_train.shape)
print("Y CV:", y_cv.shape)
print("Y Test:", y_test.shape)
In [76]:
# Label encoding- replacing the "Nos" with 0 and "Yes'" with 1
y_train.replace({'no' : 0, "yes" : 1}, inplace = True)
y_cv.replace({'no' : 0, "yes" : 1}, inplace = True)
y_test.replace({'no' : 0, "yes" : 1}, inplace = True)
Encoding Categorical Features - Two methods for encoding are 'One hot encoding' and 'Response encoding'. But i am using 'One Hot Encoding'.¶
In [77]:
# One big step before training any machine learning model is converting the categorical variables to numerical values.
# Some high efficient algos do but many machine learning algorithms cannot operate on label data directly.
# They require all input variables and output variables to be numeric.
# This means that categorical data must be converted to a numerical form.
# We perfrom this method when no natural ordering is present in the categorial values.
In [78]:
#Categorical condition to extract categorical columns
categorical_condn = data_x.dtypes == object
In [79]:
categorical_condn
Out[79]:
In [80]:
# Filtering the categorical columns and printing out
categorical_cols = data_x.columns[categorical_condn].tolist()
In [81]:
categorical_cols
Out[81]:
In [82]:
from sklearn.feature_extraction.text import CountVectorizer
def add_onehot_to_dataframe(sparse, df, vectorizer, name):
for i, col in enumerate(vectorizer.get_feature_names()):
colname = name+"_"+col
df[colname] = sparse[:, i].toarray().ravel().tolist()
return df
def OneHotEncoder(categorical_cols, X_train, X_test, X_cv = None, include_cv = False):
for i in categorical_cols:
Vectorizer = CountVectorizer(token_pattern="[A-Za-z0-9-.]+")
print("Encoding for feature: ", i)
temp_cols = Vectorizer.fit_transform(X_train[i])
X_train = add_onehot_to_dataframe(temp_cols, X_train, Vectorizer, i)
if include_cv:
temp_cols = Vectorizer.transform(X_cv[i])
X_cv = add_onehot_to_dataframe(temp_cols, X_cv, Vectorizer, i)
temp_cols = Vectorizer.transform(X_test[i])
X_test = add_onehot_to_dataframe(temp_cols, X_test, Vectorizer, i)
In [83]:
OneHotEncoder(categorical_cols, X_train, X_test, X_cv, True)
In [84]:
#Droping the categorical columns once encoding is done
X_train = X_train.drop(categorical_cols, axis = 1)
X_cv = X_cv.drop(categorical_cols, axis = 1)
X_test = X_test.drop(categorical_cols, axis = 1)
print("Shape of train:", X_train.shape)
print("Shape of CV:", X_cv.shape)
print("Shape of test:", X_test.shape)
In [85]:
X_train.info()
In [86]:
#Printing the dataset to ensure our whole dataset is numeric now.
X_train.head(5)
Out[86]:
In [87]:
pd.set_option('display.max_columns', None)
X_train.head(5)
Out[87]:
In [88]:
# Just a best practice step - saving the encoded dataset to local computer. (Can be used for small excell filtering too in needed)
data_x.to_csv("encoded_data_x.csv")
In [89]:
# Advice before building any model on this dataset is to drop the 'Duration' column , due to it's high correlation with the target variable.
Above step concludes the data processing and prepping the data for further machine learning training on the dataset.¶
In [90]:
# Creating a simple machine learning model - I've picked logistic regression model here for the reason that,
# it best fits our dataset in case of a textbook defination in my view.
# We have a binary classification model. so hence a logistic regression model.
In [91]:
# Building the model with 'Duration' column.
In [92]:
model = LogisticRegression(class_weight = 'balanced')
model.fit(X_train, y_train)
y_pred = model.predict_proba(X_test)
print("AUC score:", roc_auc_score(y_test, y_pred[:,1]))
In [93]:
print(y_pred)
In [94]:
Y_ProbList =list(y_pred)
In [95]:
# To be ran
model.score(X_train,y_train)
Out[95]:
In [ ]:
Building Logistic Regression model¶
In [86]:
data.shape
Out[86]:
In [87]:
data_new = data.drop(columns = ['ModelPrediction'])
In [88]:
data_new.shape
Out[88]:
In [89]:
contact = ({'cellular':0, 'telephone':1})
data_new['contact'] = data_new['contact'].map(contact)
In [90]:
data_new = pd.get_dummies(data_new, columns = ['job','marital','education','default',
'housing','loan','month',
'day_of_week','poutcome'], drop_first = True)
In [91]:
data_new.shape
Out[91]:
In [92]:
# Importing required scikit learn's libraries
In [97]:
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('darkgrid')
import warnings
warnings.filterwarnings('ignore')
print('.....Setup complete')
import os
In [94]:
#group data into X set and y set, where X has all features and y has the label
X = data_new.loc[:,data_new.columns != 'y']
y = data_new.loc[:,data_new.columns == 'y']
X_train,X_test,y_train,y_test = train_test_split(X,y, test_size = 0.3 , random_state = 0)
In [95]:
#print lenght of both test and train set
print('shape of X_train : ',len(X_train), '\nshape of y_train : ',len(y_train))
print('\nshape of X_test : ',len(X_test), '\nshape of y_test : ',len(y_test))
In [98]:
#Normalize the data
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
In [99]:
#train model using logistic regression
from sklearn.linear_model import LogisticRegression
clf = LogisticRegression()
clf.fit(X_train,y_train)
clf.score(X_train,y_train)
Out[99]:
In [100]:
clf.score(X_test,y_test)
Out[100]:
In [101]:
#predict X_test and save in variable y_pred
y_pred = clf.predict(X_test)
In [102]:
#Evaluate the model
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test,y_pred)
cm
Out[102]:
In [103]:
from sklearn.metrics import classification_report
print(classification_report(y_test,y_pred))
In [ ]: